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DETAILED ACTION 
Response to Amendment 

1 . In response to the office action from 2/5/2007, the applicant has submitted an 
amendment, filed 6/4/2007, amending independent claims 1 and 10 to add steps of receiving 
inputs regarding usage of different input modalities, confidence in results, and/or pragmatic 
information on modality usage, determining a target relative usage of a resource, and processing 
an input modality based on an allocated resource, while adding new claims 19 and 20 and 
arguing to traverse the art rejection based on the limitation regarding allocating resources based 
on one of: actual usage of different input modalities, confidence in results, or pragmatic 
information on modality usage (Amendment, Pages 11-12). The applicant's arguments have been 
fully considered but are moot with respect to the new grounds of rejection, necessitated by the 
amended claims and further in view of Suhm et al ("Multimodal Error Correction for Speech 
User Interfaces, 2001), 

2. In response to amended Fig. 2, the examiner has withdrawn the previous drawing 
objections. 



3. In response to amended claims 1 and 10, the examiner has withdrawn the previous claim 
objections directed towards minor informalities. 
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4. In response to amended claims 1 and 10 and corresponding arguments (Amendment, 
Pages 10-1 J), the examiner has withdrawn the previous 35 U.S.C. 1 12, second paragraph 
rejections. 

5. In response to amended claims 1 and 10 and associated arguments (Amendment, Page 

11) , the examiner has withdrawn the previous 35 U.S.C. 101 rejection. 

Response to Arguments 

6. Applicant's arguments have been fully considered but they are not persuasive for the 
following reasons: 

With respect to Claims 1 and 10, the applicant argues that Maes et al (U.S. Patent: 
6,964,023) is only concerned with keeping "a dialog flowing" and does not anticipate allocating 
a resource based on one of: the actual usage of different input modalities by a device user, 
confidence in results, or pragmatic information on input modality usage (Amendment, Pages 11- 

12) . 

In response, the examiner points out that Maes explicitly teaches resource allocation in a 
multi-modal input system. More specifically, Maes discloses a resource manager that prioritizes 
allocation of CPU cycles or input/output priorities (i.e., resources and resource allocation) (Col 
37, Lines 4-16). In Maes, resource allocation is performed based on the modality engines that 
are engaged (i.e., actual usage of the different input modalities by the user of the device) and 
modality capabilities and network path usage/delay (i.e., pragmatic information on input 
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modality usage) (Col. 37, Lines 4-16). By the disclosed "keeping a dialog flowing" argued by 
the applicant, Maes is simply referring to allocating resources to active modality engines ("the 
engines engaged for recognizing or processing a current input or output have priorities, " Col. 
37, lines 4-16) so processing speed can be appropriately maximized (i.e., avoiding delays or 
"keeping a dialog flowing") for a most utilized/active user input modality. Thus, the examiner 
notes that Maes anticipates the aforementioned claim limitation. 

The art rejection of the dependent claims is traversed for reasons similar to claims 1 and 
10 (Amendment, Page 12). In regards to such arguments, see the above response to the 
arguments directed towards claims 1 and 10. 

Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

8. Claims 1, 3, 9-10, 12, and 18-20 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Maes et al (U.S. Patent: 6,964,023) in view of Suhm et al ("Multimodal Error 
Correction for Speech User Interfaces, 2001). 

With respect to Claims 1 and 10, Maes discloses: 

A method and system for dynamically controlling usage of a resource by task entities 
respectively involved in processing different input modalities comprising: 
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Receiving inputs regarding: input mode usage by a user of the data processing device, 
modal requirements of a dialogue manager and an application or service at a bandwidth 
moderator (resource manager that receives inputs regarding engaged input modalities and 
associated dialog applications, modality capabilities/states, and network path delay, Col 37, 
Lines 4-16)\ 

Determining a target relative usage of a data-processing resource (determining required 
CPU cycles or input priorities to maintain a flowing dialog with an active application, Col 37, 
Lines 4-16); 

Wherein the relative average actual or allocated usage of the resource by the data- 
processing entities is dynamically allocated by said bandwidth moderator according to one or 
more of the following: actual usage of the different modalities by a user; confidence in the results 
of processing of each of the modalities; pragmatic information on input modality usage 
(allocating computer processing resources based on in use (active) modality engines, modality 
capabilities, and network delay, Col. 37, Lines 4-16; and Col 35, Line 54- Col 36, Line 7)\ and 

Processing at least one of the input modalities using the resource as dynamically 
allocated by the bandwidth moderator (modality engines for processing/recognizing a user input 
by utilizing a prioritized resource, Col 37, Lines 4-16). 

Maes further discloses multiple applications associated with various modalities and a 
corresponding computer processor (Col 7, Lines 40-45; Col 31, Lines 15-34; and Col 45, Lines 
17-44). 

Maes does not specifically suggest receiving a confidence score in a recognition process 
for resource allocation (i.e., "and/or confidence in a recognition process "). Suhm, however, 
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recites using confidence scores to identify recognition errors and switch to another modality (i.e., 
activating processing resources for a different input modality) (Pages 68, 71, 74-75, and 94), 

Maes and Suhm are analogous art because they are from a similar field of endeavor in 
systems having multimodal input processing. Thus, it would have been obvious to a person of 
ordinary skill in the art, at the time of invention, to modify the teachings of Maes with the 
confidence metric for resource allocation switching taught by Suhm in order to speed up 
correction of recognition errors (Abstract, Page 60). 

With respect to Claims 3 and 12, Maes further discloses: 

The resource is processing power (computer processing power allocated to active 
modality engines, Col. 37, Lines 4-16). 

With respect to Claims 9 and 18, Maes further recites: 
Controlling the allocation of the resource between task entities (allocating computer processing 
resources based on in use (active) modality engines associated with a particular application, 
Col 37, Lines 4-16; and Col. 35, Line 54- Col. 36, Line 7). 

With respect to Claims 19-20, Maes discloses resource allocation based on data received 
at a resource manager comprising: engaged input modalities (actual usage of the different input 
modalities) and modality capabilities/states and network path delay (pragmatic information on 
input modality usage), while Suhm discloses using confidence scores to identify recognition 
errors and switch to another modality, as both applied to Claims 1 and 10. Also, Maes only 
requires only one of the aforementioned items for resource allocation at a resource manager (Col 
37, Lines 4-16). 
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9. Claims 2, 4-8, 11, 13-17 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Maes et al in view of Suhm et al and further in view of Bridger et al (WO 01/35575 A2). 

With respect to Claims 2, 4, 11, and 13, Maes in view of Suhm discloses the system and 
method for allocating computer processing resources based on modality usage as applied to 
Claims 1 and 10. Maes further discloses multiple applications associated with various modalities 
(Col. 7, Lines 40-45; and Col 31, Lines 15-34). Maes in view of Suhm does not explicitly 
suggest resources comprising communication bandwidth and memory, however Bridger 
discloses such resource allocation (Page 1, Line 32- Page 2, Line 10; and Page 7, Lines 15-29). 

Maes, Suhm, and Bridger are analogous art because they are from a similar field of 
endeavor in processing for multiple input data types. Thus, it would have been obvious to a 
person of ordinary skill in the art, at the time of invention, to modify the teachings of Maes in 
view of Suhm with the resource management scheme taught by Bridger in order to prevent a 
server from overload while not wasting its usage (Bridger, Page 1, Lines 19-30). 

With respect to Claims 5 and 14, Bridger further discloses processing resources 
determined based on an input modality, wherein a single processing resource can be assigned to 
different modality types (Page 7, Lines 7-29). 

With respect to Claims 6 and 15, Bridger further recites the adjustment of related 
resources (Page 7, Line 30- Page 8, Line 4). 

With respect to Claims 7 and 16, Bridger further recites an allocation of resources based 
on a modality (Page 7, Lines 7-14) and further allocation for similar modality types (Page 10 t 
Lines 3-12). 



Application/Control Number: 10/607,577 Page 8 

Art Unit: 2626 

With respect to Claims 8 and 17, Bridger further recites evenly dividing resources for all 
high priority tasks (Page 9, Lines 11-23) and further allocation for similar modality types (Page 
10, Lines 3-12). 

Conclusion 

10. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

1 1 . The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure: 

Brittan et al (U.S. Patent: 6, 725,199)- discloses a system that changes the type of 
synthesis engine in the event of low confidence scores. 
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Bourguet et al (U.S. Patent Application Publication: 2002/0178344)- teaches a 
multimodal management system that switches to a different input modality in response to low 
confidence scores. 

1 2. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to James S. Wozniak whose telephone number is (571) 272-7632. 
The examiner can normally be reached on M-Th, 7:30-5:00, F, 7:30-4, Off Alternate Fridays. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Patrick Edouard can be reached at (571) 272-7603. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



James S. Wozniak 
8/2/2007 




